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ABSTRACT 

Flavour changing neutral decays of the top quark are know to be extremely 
suppressed in the SM. This is especially so for the top quark decay into the 
SM Higgs boson, whose rate is less than 10^^^. However, it turns out that the 
decay modes t ch, with h = h^,H^,A^ any of the MSSM neutral Higgs 
bosons, can be much more gifted. In particular, the rate into the lightest 
CP-even Higgs boson - which is an accessible decay mode across the whole 
MSSM parameter space - is generally higher than the decay into glue, t —>■ eg, 
even though the latter can also be considerably augmented in the MSSM. Our 
general conclusion is that the Higgs channel t — >■ c can be the "gold-plated" 
top quark FCNC decay mode in the MSSM with rates that can reach lO"*^, 
i.e. already at the visible level for both LHC and LC. 



^Updated talk presented in: International Workshop on Linear Colliders, Sitges, April 28-May 5, 
1999; to appear in the proceedings. 



1 Introduction 



At the tree-level there are no FCNC processes in the SM, and at one-loop they are induced 
by charged-current interactions, which are GIM-suppressed. In particular, FCNC decays 
of the top quark into gauge bosons {t ^ cV; V = 'y, Z,g) are very unlikely, with maximum 
rates of order 10~^^ for the photon, slightly above 10~^^ for the Z'-boson, and at most 
10"^" for the gluon channel 0. Even more dramatic is the situation with the top quark 
decay into the SM Higgs boson, t — cHsm, which has recently been recognized to be 
much more disfavored than originally thought BR{t cHsm) = 1 ■ 10~^^ — 4 ■ 10~^^ 
{Mz < Mh < 2 M\y) 0. This rate is far out of the range to be covered by any presently 
conceivable high luminosity machine. On the other hand, the highest SM rate, namely 
that of t — > c(7, is still 5 orders of magnitude below the feasible experimental possibilities 
at the LHC. All in all detection of FCNC decays of the top quark at visible levels (viz. 
BR{t — > cX) > 10~^ — 10~^) by any of the future high luminosity colliders round the 
corner (especially LHC and LC) seems doomed to failure in the absence of new physics. 

Thus the possibility of large enhancements of some FCNC rates up to visible lev- 
els, particularly within the context of general two-Higgs-doublet models (2HDM) and 
especially in the Minimal Supersymmetric Standard Model (MSSM) should be very 
welcome. In the LHC, for example, the production of top quark pairs will be very high: 
a{tt) = 800 pb - roughly two orders of magnitude larger than that of the Tevatron II 
at ^/s = 2TeV. In the so-called low-luminosity phase {10^^ cm~^ s~^) of the LHC one 
expects ten million t t-pairs per year ||^ . And this number will be augmented by one order 
of magnitude in the high-luminosity phase (10^^ cm^^s^^). As for a future e^e~ linear 
collider running at e.g. ^/s = 500 GeV, one has a smaller cross-section a{tt) = 650 fb 
but a higher luminosity factor ranging from 5 x 10^^ cm~^s~^ to 5 x 10'^^ cm~^s~^ and of 
course a much cleaner environment [^]. Thus, with datasets from LHC and LC increas- 
ing to several 100/6~^/year in the high-luminosity phase, one should be able to collect 
enough statistics (perhaps some few hundred to few thousand events) from the combined 
output of these machines enabling us to perform an efficient study of rare top quark decays 
beyond the SM. A detailed presentation of this work is given in Ref. 0. 

2 Relevant decays and Lagrangians in the MSSM 

The Higgs channels in the MSSM are especially promising. They comprise the top quark 
decays into the two CP-even ("scalar") states and the CP-odd ( "pseudoscalar" ) state of 
the Higgs sector of the MSSM |[, 

t^ch {h = h\H\A^). (1) 

Worth emphasizing is the fact that in the MSSM (in contrast to the SM or the uncon- 
strained 2HDM) at least one of these decays (viz. t —>■ ch^) is always possible, for in 
the MSSM there is an upper bound on the mass of the lightest CP-even Higgs boson, 
Mho < 135 GeV which is below the top quark mass. Moreover, for a sufficiently light 
pseudoscalar mass, M^o < m^, all three decays (|1|) are in principle possible, if the SUSY 
masses are not that high so as to induce too large positive corrections to Mj^o. Prelimi- 
nary studies of the decays (|1]) were presented in 0, but they did not include the one- loop 
Higgs mass relations and moreover they missed some of the most relevant effects. 
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Figure 1: One-loop SUSY-QCD diagrams for the decay t —>■ ch: (a) vertex diagram, (b) 
mixed t — c self-energy. U{a,i3} stand for mass-eigenstate up- type squarks of any generation. 

We report here also on the MSSM rate of the most favored FCNC top quark decay in 
the SM, namely the top decay into glue 

t^cg. (2) 
This channel has been extensively studied in the literature, both in the SM |^ and 



in the MSSM |jTO|, [TT|. However, we include a simultaneous numerical analysis of it in 
order to use it as a fiducial mode with which to better assess the size of our results on the 
Higgs channels (|l]). As for the remaining FCNC decay modes, namely t — > 0(7, Z), they 
have also been evaluated in the MSSM, but in spite of some enhancements the general 
conclusion is that they are hopeless |llO| , |T2[ . 



The basic interactions responsible for the enhancements lie both in the QCD sector and 
in the electroweak (EW) sector of the MSSM. The SUSY-QCD interactions are described 
by the following Lagrangian, in the mass-eigenstate basis: 



-^SUSY-QCD - "^c [R*5a Pl - Rla Pr\ Q*a,i Kj 



- ^ Pl - Rla Pr\ 

- ^ rc [Rla Pl - Pr] + h.c. . (3) 

V'f stands for the gluino spinor, are the S'f/(3)c Gell-Mann matrices and Pl,r the chiral 
projector operators. The 6x6 rotation matrices R'^'^^ are needed to diagonalize the squark 
mass matrices in (flavor) x (chiral) space as follows (notation as in [^): 

R^^^^MlR = Mlj, = diag{ml,...,ml} {q^u,d), (4) 

where -^^^j-, is the square mass matrix for squarks in the EW basis {q'a), with indices 

a = 1, 2, 3, . . . , 6 = ul, Ufi, cl, . . . ,ifi for up-type squarks, and a similar assignment for 
down-type squarks. The intergenerational mixing terms leading to gluino-mediated FCNC 
couplings lie in the off-diagonal entries of the mass matrices. However, in order to prevent 
the number of parameters from being too large, we have allowed (symmetric) mixing mass 
terms only for the left-handed (LH) squarks. This simplification is often used in the MSSM 
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Figure 2: One-loop SUSY-EW diagrams for the decay t ch {h = h^,H^,A'^). Here d 
{d{a,b}) represent mass-eigenstate down type quarks (squarks) of any generation. 



and it is justified by Renormalization Group (RG) analysis |T3| . Following this practice, 
we introduce flavor-mixing coefficients 6ij in the LL block of the 6x6 squark mass matrix 
(namely the one involving only LH fields of any flavor) as follows: 



LL)ij 



5ij rrii rrij 



^ j) 



(5) 



where mj is the mass of the left-handed zth squark, and ml- is the mixing mass matrix 
element between generations i and j. Therefore, if the coefficients 5ij are non- vanishing, 
for some i ^ then the structure of the diagonalizing matrices i?'-''-' deflned above must 
necessarily lead to gluino-mediated tree-level FCNC between quarks and squarks in the 
SUSY-QCD Lagrangian (^). This scenario can be generalized by further introducing 
FCNC interactions on the right-handed (RH) block of the mass matrix, but this hypothesis 
entails an unnatural departure from the RG expectations and moreover it is not necessary 
to achieve the visible rates [0 . 

On the other hand, the leading EW interactions appear through the possibility of large 
Yukawa couplings (normalized with respect to the SU{2)l gauge coupling): 

K _ rriu . _ hd 

9 



A„ = 



rrid 



(6) 



V2Mwsmf3 ' g V2Mwcos(3 

For the computation of the EW effects we work in the approximation that the squark 
mass matrices diagonalize whith the same matrix elements than the quarks, the standard 
CKM-matrix. Although we will not spell out here the structure of the various relevant 
interactions in the EW sector, let us for the sake of illustration to quote the interaction 
Lagrangian involving charginos, quarks and squarks in the mass-eigenstate basis (notation 
as in 



0]) 



C 



udx-' 



{d,u) p /i{d,u) p 

+ai -|- ^-ai 



U + h.c. 



(7) 



d=d,s,b 
u=u,c,t 
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Vud is the standard CKM-matrix element, and the couphng matrices are defined as 

A(d,u) _ p('^)*T/* \ TD{d)*-,r* ,(d,u) _ dW* \ TT ( Q\ 



with Vij and Uij the rotation matrices of the chargino sector [113. Here A„, are the 



up-hke and down-hke Yukawa couphngs (^. Its significance is determined by the value 
of the parameter tan/3 = V2/V1 [ffl. At large tan/5 one expects that the EW interactions 



may play here a significant role as in the case of top quark decays into charged Higgs []T5|. 
From these interaction Lagrangians, the Feynman diagrams responsible for the FCNC 
decays can be depicted in Fig.l (SUSY-QCD effects) and Fig. 2 (SUSY-EW). One 
may draw similar diagrams for the decay @). 



3 Numerical Analysis and Discussion 

We use the fiducial ratios 

B{t ^ch)^ nt^ch) ^ ^ nt^cg) ( ) 

to carry out our numerical analysis. 

These ratios are not the total branching fractions BR{t cX) for the FCNC decay 
modes under study {X = h,g), as there are many other channels that should be added 
up to the denominator of (P) in the MSSM, if kinematically allowed. However, for better 
comparison with previous analyses of FCNC top quark decays, the ratios (|^) should suffice 
to assess the experimental viability of the FCNC decays under consideration. 

After an exhaustive scanning of the MSSM parameter space we find the following 
results [0. The most important source of FCNC enhancement comes from the SUSY- 
QCD sector, whose numerical effect hinges on the values of the flavour-mixing coefficients 
Sij in eq.(|^). The present experimental bounds on these quantities depend on the value 
of the squark masses and read as follows |jl6| : 





< 




< 


(^23 


< 



.1 rric/ 500 GeV , 
.098 ^TUu mi/500 GeV , 

8.2 mcmi/ {500 GeVf . (10) 

In using these bounds we make use of SU (2) gauge invariance to transfer the experimental 
information known from the down-quark sector (for example from BR{b — s 7), where 
the bound on ^23 is obtained) to the up-quark sector. 

With a SUSY mass spectrum of a few hundred GeV, we flnd that the different con- 
tributions to the Higgs channels are typically of the order 



B^^^^-^^{t^ch) ~ 10-« , 

However, by stretching out a bit more the range of parameters one can reach (for some 
of the decays) 

^SUSY-EW^^^^;^) ^ lxlO-% 

^susY-QCD^^_^^^) ^ 5x10-^ (12) 
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Figure 3: (a) Maximum value of B{t — * ch), obtained by taking into account only the 
SUSY-EW contributions, as a function of M^o ; (b) as in (a) but taking into account only 
the SUSY-QCD contributions; and (c) maximum value of B{t —>■ eg) a.s a. function of 
the intergenerational mixing parameter ^23 in the LH sector. In all cases the scanning for 
the rest of parameters of the MSSM has been performed within the phenomenologically 
allowed region. 



The difference of at least two orders of magnitude between the SUSY-EW and SUSY- 
QCD contributions makes unnecessary to compute the interference terms between the two 
sets of amplitudes. 



We have obtained the maximum rates (|T2D from a general search in the MSSM pa- 
rameter space within the 1 TeV mass region and respecting the experimental constraints, 



in particular the relations ([T0|) . In Fig. 3 we present the maximum values that can be 
reached by B{t — > cX) for each of the processes presented. In Figs. 3a and 3b we show 
the maximized B{t ^ ch) as a function of the pseudoscalar Higgs boson mass by taking 
into account only the SUSY-EW contributions and the SUSY-QCD contributions respec- 
tively. Perhaps the most noticeable result is that the decay into the lightest MSSM Higgs 
boson {t ch^) is the one that can be maximally enhanced and reaching values of order 
B{t — >■ ch^) ~ 10~^ that stay fairly stable all over the parameter space. The reason for 
this dominance is that the decay t — > c is the one which is more sensitive to the trilinear 
coupling At, a parameter whose natural range reaches up to about ITeV. 

We have also studied the optimal FCNC rates of the gluon channel in the MSSM. In 
Fig. 3c we have plotted the maximum value of B{t eg) a.s a function of ^23 after scanning 
for the rest of the MSSM mass parameters within the 1 TeV range. Under the RG-based 
assumption of only mixing in the LH sector, one has ijSUSY-QCD^^ ,-(7) < 10~^. Our 
results on this channel are compatible with the recent analysis of Ref. in contrast to 
that of Ref. O 
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To assess the discovery reach of the FCNC top quark decays in the next generation of 
accelerators we take as a guide the estimations that have been made for gauge boson final 
estates |T^. Using the information mentioned in Sect.^ and assuming that all the FCNC 



decays t ^ cX {X = V,h) can be treated similarly, we roughly estimate the following 
sensitivities for 100 fb^^ of integrated luminosity: 



LHC -.Bit - 


cX) 


> 


5 X 10"^ 


LC -.Bit - 


cX) 


> 

r\-/ 


5 X 10~^ 


TEV33 - 


cX) 


> 


5 X 10^^ 



(13) 

Therefore, LHC seems to be the most suitable collider where to test this kind of phenom- 
ena. The LC is limited by statistics (due to much smaller top quark cross-section) but in 
compensation every collected event is clear-cut. So this machine could eventually be of 
much help, especially if we take into account that it could deliver 500 fb~^ per year[^]. 
The situation with the Tevatron, however, is much more gloomy, since the required FCNC 
rates of ~ 10~^ cannot be attained unless we artificially search in some remote (unnatural) 
corner of the parameter space. 

To conclude, the FCNC decays of the top quark are such rare events in the SM 
(especially the top quark decay into the Higgs boson) that their observation at detectable 
levels should be interpreted as an extremely robust indication of new physics. The effective 
tagging of FCNC top quark decays in the major accelerators round the corner could well 
be a first step into discovering SUSY. From our analysis we find the following MSSM 
maximum rates for the most favorable modes: 

5 X 10-6 < B{t ^ c(7)^ax < B{t ^ c/i°)„,ax < 5 X 10-^ (14) 

In both decays the dominant effects come from SUSY-QCD. However, it should not be 
undervalued the fact that the maximum electroweak rates for t ch can reach the 10"^ 
level, i.e. on the verge of being detectable. 
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